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(57) ABSTRACT 

The distributed system includes a database server, an admin- 
istration module, a process evaluation module, and a data 
display module. The administration module is in communi- 
cation with the database server for providing observation 
criteria information to the database server. The process 
evaluation module is in communication with the database 
server for obtaining the observation criteria infonnation 
from the database server and collecting process data based 
on the observation criteria information. The process evalu- 
ation module utilizes a personal digital assistant (PDA). A 
data display module in communication with the database 
server, including a website for viewing collected process 
data in a desired metrics form, the data display module also 
for providing desired editing and modification of the col- 
lected process data. The connectivity established by the 
database server to the administration module, the process 
evaluation module, and the data display module, minimizes 
the requirement for manual input of the collected process 
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PROCESS EVALUATION DISTRIBUTED 
SYSTEM 

STATEMENT REGARDING FEDERALLY 
SPONSORED RESEARCH OR DEVELOPMENT 5 

The invention described herein was made in the perfor- 
mance of work under NASA Contract Number NAS9- 
20000, subcontract number 1970483303, and is subject to 
the provisions of Section 305 of the National Aeronautics to 
and Space Act of 1958(72 Stat. 435: 42 U.S.C. 2457). 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 15 

The present invention relates to process surveillance and 

more particularly to a distributed system for analyzing data 
collected by reviewing work documents, inspecting work 
areas and passively observing work in progress. 

2. Description of the Related Art 20 

Present process surveillance systems typically have prob- 
lems related to the constant manual input of data. Process 
surveillance systems generally utilize clipboards or some 
form of electronic device to collect process data. The 
collected process data then has to be manually entered into 25 
a spreadsheet or database. Once the data is entered, it has to 
be organized so that it can be charted and graphed. Reports 
need to be created manually as well. The manual entering 
and creation of charts, graphs, and reports takes considerable 
man-hours to accomplish. 30 

In order to share the collected process data, a copy of the 
file needs to be exchanged. Any changes to the original file 
does not appear in copies of the data file, thus creating 
discrepancies among the collected data. 

To collect process data, observation criteria information is 35 
needed to guide the inspector in what to observe. Changes 
to the observation criteria information often take consider- 
able time to become available to the inspector. Time delays 
also occur when requirements or changes to the process take 
place. 40 

U.S. Pat. Nos. 5,764,509 and 6,181,975, both issued to 
Gross, et al., and both entitled, “Industrial Process Surveil- 
lance System” disclose a system and method for monitoring 
an industrial process and/or industrial data source. The 
system includes generating time varying data from industrial 45 
data sources, processing the data to obtain time correlation 
of the data, determining the range of data, determining 
learned states of normal operation and using these states to 
generate expected values, comparing the expected values to 
current actual values to identify a current state of the process 50 
closest to a learned, normal state; generating a set of 
modeled data, and processing the modeled data to identify a 
data pattern and generating an alarm upon detecting a 
deviation from normalcy. This system does not appear to be 
random in any fashion and may therefore be inadequate for 55 
statistical analysis purposes. 

U.S. Pat. No. 5,765,138, issued to Aycock, et al., entitled, 
“Apparatus and Method for Providing Interactive Evalua- 
tion of Potential Vendors”, discloses an arrangement for 
providing an interactive evaluation of suppliers as proposed 60 
vendors for a project. A supplier evaluation system com- 
prises a database storing a plurality of maturity requirements 
and recognized quality standards, and a main processing 
system for compiling selected standards and quality maturity 
requirements in accordance with project objectives. The 65 
project requirements are supplied by a co mmu nication net- 
work to a supplier in the form of an interactive supplier 
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self-evaluation system. Alternatively, the supplier self- 
evaluation system may be provided to a supplier on a 
CD-ROM. The supplier self-evaluation system is arranged 
to include a plurality of objective questions corresponding to 
the selected maturity requirements. A supplier may selec- 
tively access local database files on the CD-ROM for 
information regarding the selected maturity requirements, or 
may remotely access the supplier evaluation system data- 
bases for supplemental information. After uploading the 
supplier responses to the supplier evaluation system, a 
supplier maturity level is calculated. An on-site supplier 
audit is thereafter conducted to confirm supplier responses 
and to obtain any additional information. Supplier approval 
is dependent upon a minimum supplier maturity level based 
upon the scored supplier responses to the maturity questions 
and the on-site audit. This system, like the Gross et al system 
described above, does not appear to be random. Further- 
more, it is focused relative to the evaluation of vendors. 

SUMMARY 

The present invention is a distributed system for observ- 
ing processes. In a broad aspect it includes a database server, 
an administration module, a process evaluation module, and 
a data display module. The administration module is in 
communication with the database server for providing 
observation criteria information to the database server. The 
process evaluation module is in communication with the 
database server for obtaining the observation criteria infor- 
mation from the database server and collecting process data 
based on the observation criteria information. The process 
evaluation module utilizes a personal digital assistant 
(PDA). A data display module in communication with the 
database server, including a website for viewing collected 
process data in a desired metrics form, the data display 
module also for providing desired editing and modification 
of the collected process data. The connectivity established 
by the database server to the administration module, the 
process evaluation module, and the data display module, 
minimizes the requirement for manual input of the collected 
process data. 

The system preferably randomly determines when to 
collect the process data and therefore has statistical signifi- 
cance. This is important for determining the efficiency of the 
process. 

Operating costs remain as one of the most significant 
expenditures for most companies today. If operating costs 
could be reduced even by a small percent, the cost savings 
for a company would be very substantial. The system of the 
present invention can thus greatly reduce operating costs. 
Using the present system, every aspect of the process can be 
scrutinized to determine if it lacks consistency. Any aspect 
of the process found lacking consistency can be singled out 
and improved. Process assessment and improvement can 
lower operating costs in many ways. The present invention 
can reduce costly unexpected incidents through greater 
employee awareness and attention to detail. It can also be 
used to increase safety. Improving the process can also lower 
turn around time and lead to a decrease in overhead, both of 
which can increase a company’s profits. 

Other objects, advantages, and novel features will become 
apparent from the following detailed description of the 
invention when considered in conjunction with the accom- 
panying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS The data display module 26 preferably includes the capa- 

bility of generating a random schedule for determining when 
FIG. 1 is schematic flow diagram of a preferred ernbodi- to collect process data. For example, this may be done at the 
ment of the present invention. beg inni ng of every week. 

FIG. 2 is a flow diagram of the data synchronization 5 Referring now to FIG. 2, the data synchronization sub- 
sub-module of the process evaluation module of the present module 22 is shown in relation with user functions and the 
invention. cradling/uncradling of the PDA. The sub-module 22 

FIG. 3 is a flow diagram of the data collection sub-module includes a computer implementable program for providing a 

of the process evaluation module of the present invention. series of steps. First, the PDA is cradled, as noted by block 

The same parts or elements throughout the drawings are to 28. A synchronization program is executed (30) for obtain- 
designated by the same reference characters. ing of the observation criteria information 16 from the 

database server 12. There is an attempt to download the 
DETAILED DESCRIPTION OF THE observation criteria information 16 from the database server 

INVENTION 12. There is a determination 32 made as to if there is any 

15 collected process data 18. 

Referring now to the drawings and the characters of Observation criteria information 16 is downloaded to the 
reference marked thereon, FIG. 1 illustrates a preferred data collection sub-module 24 if there is no collected 

embodiment of the distributed system of the present inven- process data 18, as indicated by numeral designation 34. In 

tion, designated generally as 10. The distributed system 10 such an instance, the PDA is uncradled (36), data is then 

includes a database server 12 that may use, for example, any 20 collected (38), and the PDA is then cradled (40). A synchro- 
structured query language (SQL) compliant database server nization is then performed, as indicated by numeral desig- 

(Oracle, Microsoft SQL Server, etc.). nation 42. The collected process data 18 is then uploaded to 

An administration module 14 is in communication with the database server 12 (see block 44). 
the database server 12 for providing observation criteria If there is collected process data 18 the user is prompted 

information 16 to the database server 12. The administration 25 to upload the collected process data 18. The PDA is then 
module 14 includes a computer implementable Administra- uncradled (46). 

tion Program for providing the functions of: entering, edit- Referring now to FIG. 3, the data collection sub-module 
ing and deleting observation criteria information 16; grant- 24 is illustrated. Sub-module 24 includes a computer imple- 
ing access to users; and, editing collected process data 18. mentable Data Collection Program (DCP). The DCP 24 

The observation criteria information 16 may be, for 30 prompts a user to login (48) and to select (50) at least one 

example, checklists, error codes, operating procedure docu- Active Work Area. Typically several Active Work Areas are 

ments, specifications, etc. Collected process data 18, may selected. An active work area is a location in which pro- 
be, for example, acceptability of the checklists, error codes cessing work is being conducted, usually somewhere, for 

associated with any failures, i.e. unacceptable checklists, example, on a shop floor. 

etc. The Administration Program is a menu driven interface 35 One of the selected Active Work Areas is preferably 
to the database server. It allows the administrator to edit the randomly chosen (52) and the observation criteria informa- 
collected process data and grants access to users. It may use tion 16 is also preferably randomly chosen (54). Thus, the 

a commercially available programming language such as user is allowed to gather the collected process data 18. The 

Visual Basic, or C++. DCP 24 decides where (i.e. the Work Area), what to observe, 

A process evaluation module, designated generally as 20, 40 and how to observe it. 
is in communication with the database server 12 for obtain- The acceptability of the collected process data 18 is 
ing observation criteria information 16 from the database recorded based upon the observation criteria information 16, 

server and collecting process data 18 based on the observa- as shown by block 56. Thus, the user is able to utilize his 

tion criteria information 16. The process evaluation module PDA to record acceptability findings and to input comments, 

utilizes a personal digital assistant (PDA). The PDA may be, 45 as shown by blocks 58, 60. 

for example, a Palm Pilot or Pocket PC compatible device. A determination (62) is made as to whether additional 

The process evaluation module 20 preferably includes a observations may be made to gather additional collected 

data synchronization sub-module 22, for obtaining the process data. Such additional gathering may be made by 

observation criteria information 16 from the database server; repeating the above steps. 

and, a data collection sub-module 24 for collecting the 50 The following illustrates the utilization of the present 
process data based on the observation criteria information invention relative to rocket engine processing: At the begin- 

and for providing the collected process data 18 to the data ning of the week, usually on Mondays, a Process Evaluation 

synchronization sub-module 22 for uploading to the data- Observation Schedule is randomly generated for Space 

base server 12. Sub-modules 22 and 24 will be discussed in Shuttle Main Engine (SSME) processing. This schedule 

detail below. 55 consists of 8 two-hour observation periods evenly divided 

A data display module 26 is in communication with the among first and second shift. During the week when it is 

database server. The data display module 26 includes a time to perform an observation, the PDA is cradled and a 

website for viewing collected process data 18 in a desired copy of the observation criteria information 16 is down- 

metrics fomi. The data display module 26 also provides a loaded. This ensures that the PDA has the most up to date 

desired editing and modification of the collected process 60 information. After the download is complete, the PDA is 
data 18. The views preferably include filtered views includ- uncradled and the observation period can begin. The Data 

ing graphs, charts, plots, and tables. Collection Program (DCP) 24 is started. The DCP 24 

The data display module 26 allows users to edit their own prompts the inspector to login (48) to the system and select 

collected process data 18. This gives the user access to (50)Active Work Areas. Active Work Areas are the locations 

resources for researching process data findings. This also 65 around the Kennedy Space Center (KSC) in which SSME 
gives the user access to a keyboard and spell checking processing is currently taking place. Once the inspector is 

capabilities . logged in and the Active Work Areas have been set, the DCP 
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24 randomly selects (52) one of the Work Areas for the 
inspector to perforin his/her observation. 

The DCP 24 also randomly selects what the inspector will 
observe; in the case of SSME processing this will include 
in-process work, documentation, and work area compliance. 
If there are multiple processing tasks being performed at the 
randomly selected Work Area, the DCP (24) will randomly 
select one of the tasks. The DCP 24 presents the inspector 
with the randomly selected observation criteria information 
(54), which is essentially a checklist to be used as a guideline 
for performing the observation. In SSME processing, the 
checklist will include items to ensure proper safety, the 
established procedures are being followed, and the proper 
documentation is being used. Each checklist item’s accept- 
ability is determined (56). If a checklist Item is determined 
to be unacceptable, further steps are taken to document the 
violation (58). Error codes that have been previously entered 
through the Administration Program are now available for 
assignment to the violation. The inspector can also deter- 
mine which regulatory document has been violated. When 
the observation is completed, the inspector has the ability to 
enter any additional comments into the PDA (60). These 
steps are repeated until the two-hour observation period is 
over (62). Once the two-hour observation period is com- 
plete, the inspector cradles the PDA and uploads the col- 
lected process data (40, 42, 44). The collected process data 
is now available to the inspector to edit through the website 
26. This gives the inspector access to a keyboard and spell 
check capabilities. Once the inspector is satisfied with the 
edited collected process data, the inspector saves the 
changes and the collected process data becomes available 
for use in metrics. The collected process data can now be 
categorized in many different ways to try and determine the 
life and efficiency of the process. The collected process data 
18 can also be entered into reports and used for presenta- 
tions. 

Although the present system 10 has been described with 
application to rocket engine processing, the system was 
designed to be generic so that it could be used in any 
manufacturing or processing environment. For example, it is 
particularly adaptable for use within the automotive, aero- 
space, airline, and other manufacturing industries. The sys- 
tem provides instant and ongoing feedback regarding the life 
of the observed process. The cost for this system should be 
considerably less than that charged by auditing agencies. 
The user interfaces are very user friendly and easy to 
understand. For example, the programs used in this system 
preferably provide pull-down menus, checkboxes, and 
simple displays. 

Obviously, many modifications and variations of the 
present invention are possible in light of the above teach- 
ings. It is, therefore, to be understood that within the scope 
of the appended claims, the invention may be practiced 
otherwise than as specifically described. 

The invention claimed is: 

1. A distributed system for observing processes, compris- 
ing: 

a database server; 

an administration module in communication with said 
database server for providing observation criteria infor- 
mation to said database server; 

a process evaluation module in communication with said 
database server for obtaining said observation criteria 
information from said database server and collecting 
process data based on said observation criteria infor- 
mation, said process evaluation module utilizing a 
personal digital assistant (PDA); and 
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a data display module in communication with said data- 
base server, including a website for viewing collected 
process data in a desired metrics form, said data display 
module also for providing desired editing and modifi- 
5 cation of said collected process data; 

wherein the connectivity established by said database 
server to said administration module, said process 
evaluation module, and said data display module, mini- 
mizes the requirement for manual input of said col- 
10 lected process data; 

wherein said process evaluation module, comprises: 
a data synchronization nib-module for said obtaining of 
said observation criteria information from said data- 
base server; and 

15 a data collection sub-module for said collecting of said 
process data based on said observation criteria infor- 
mation and for providing said collected process data to 
said data synchronization sub-module for uploading to 
said database sewer; and 

20 wherein said data synchronization sub-module comprises 
a computer implementable program for providing the 
steps of: 

executing a synchronization program for said obtaining of 
said observation criteria information from said data- 
25 base server; 

attempting to download said observation criteria informa- 
tion from said database server determining if there is 
any of said collected process data; 
downloading said observation criteria information to said 
30 data collection sub-module if there is no collected 
process data) and providing a synchronization, said 
collected process data being uploaded to said database 
server; 

prompting a user to upload said collected process data if 
35 there is collected process data; and 

uploading said collected process data obtained from said 
downloading and prompting steps if there is collected 
process data. 

2. The distributed system of claim 1, wherein said data 
40 collection sub-module, comprises a computer implement- 
able Data Collection Program for providing the steps of: 

a) prompting a user to login; 

b) prompting die user to select at least one Active Work 
Area; 

45 c) randomly choosing one of said selected Active Work 
Areas and randomly choosing said observation criteria 
information, thus allowing a user to gather said col- 
lected process data; 

d) recording the acceptability of said collected process 
50 data based upon said observation criteria information, 

thus allowing the user to record acceptability findings 
and to input comments; and, 

e) determining whether to gather additional collected 
process data, using steps c)-d), above. 

55 3. The distributed system of claim 1, wherein said admin- 

istration module, comprises a computer implementable 
Administration Program for providing the functions of: 
a) entering, editing and deleting said observation criteria 
information; 

60 b) granting access to users; and, 

c) editing said collected process data. 

4. The distributed system of claim 1, wherein said data 
display module: 

a) views said collected process data in said desired 
65 metrics form, comprising filtered views including 

graphs, charts, plots, and tables; and, 

b) allows users to edit their own collected process data. 
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5. The distributed system of claim 1, wherein said process 
evaluation module randomly determines when to collect said 
process data. 

6. The distributed system of claim 1, wherein said data 
display module includes means for determining when to 5 
collect said process data. 

7. The distributed system of claim 1, wherein the process 

evaluation module randomly chooses, from one or more 
active work areas, a work area for the collection of the 
process data. to 

8. The distributed system of claim 1, wherein the process 
evaluation module randomly chooses, the observation cri- 
teria information to be used for the collection of the process 
data. 

9. A distributed system for observing rocket engine pro- 15 
cessing, comprising: 

a database server; 

an administration module in communication with said 
database server for providing observation criteria infor- 
mation to said database server; 20 

a process evaluation module in communication wit said 
database server for obtaining said observation criteria 
information from said database sewer and collecting 
process data based on said observation criteria infor- 
mation, said observation criteria information compris- 25 
ing checklists for assuring proper safety and procedural 
compliance with rocket engine processing regulations, 
said process data comprising information regarding the 
acceptability of said checklists, said process evaluation 
module utilizing a personal digital assistant (PDA); and 30 
a data display module in communication with said data- 
base server, including a website for viewing collected 
process data in a desired metrics form, said data display 
module also for providing desired editing and modifi- 
cation of said collected process data; 35 

wherein the connectivity established by said database 
server to said administration module, said process 
evaluation module, and said data display module, mini- 
mizes the requirement for manual input of said col- 
lected process data; 


10. The distributed system of claim 9, wherein said data 
collection sub-module, comprises a computer implement- 
able Data Collection Program for providing the steps of: 

a) prompting a user to login; 

b) prompting the user to select at least one Active Work 
Area; 

c) randomly choosing one of said selected Active Work 
Areas and randomly choosing said observation criteria 
information, thus allowing a user to gather said col- 
lected process data; 

d) recording the acceptability of said collected process 
data based upon said observation criteria information, 
thus allowing the user to record acceptability findings 
and to input comments; and, 

e) determining whether to gather additional collected 
process data, using steps c)-d), above. 

11. The distributed system of claim 9, wherein said 
administration module, comprises a computer implement- 
able Administration Program for providing the functions of 

a) entering, editing and deleting said observation criteria 
information; 

b) granting access to users; and, 

c) editing said collected process data. 

12. The distributed system of claim 9, wherein said data 
display module: 

a) views said collected process data in said desired 
metrics form, comprising filtered views including 
graphs, charts, plots, and tables; and, 

b) allows users to edit their own collected process data. 

13. The distributed system of claim 9, wherein said 
process evaluation module randomly determines when to 
collect said process data. 

14. The distributed system of claim 9, wherein said data 
display module includes means for determining when to 
collect said process data. 



